The present invention relates to display techniques and, in particular, to techniques for displaying musical information in a three-dimensional environment.
Music is often defined as xe2x80x9csound organized in time.xe2x80x9d Generally speaking, a musical composition is essentially a specified set of musical data parameters and their variation over time. Music notation provides a way for a composer to represent these time-varying musical data visually, such that a musician can interpret the data as instructions and perform the composition as audible music.
Several systems exist for visually displaying musical data. One such system is known as xe2x80x9ccommon practice notationxe2x80x9d or xe2x80x9cCPN.xe2x80x9d FIG. 1A depicts an example of music displayed using common practice notation. The musical time axis is represented as a spatial path that proceeds from left to right along the page. The musical data are represented as graphical note symbols plotted on a staff along this path. These note symbols indicate the individual timing, duration, and pitch for each of a series of musical tones. A note""s timing and duration is indicated by its horizontal position along the staff, as well as by its type of note head and stem. A note""s pitch is indicated by its vertical position on the staff. CPN also includes many additional symbols (not shown in FIG. 1A) for indicating various other musical properties such as volume, articulation, etc.
Another notation system is known as xe2x80x9cpiano roll notationxe2x80x9d or xe2x80x9cPRN.xe2x80x9d FIG. 1B depicts an example of music displayed in piano roll notation. The musical time axis is represented as a spatial path that proceeds from top to bottom along the page. Musical data are represented as graphical note symbols plotted along this path. The note symbols (generally vertical bars) indicate the timing, duration, and pitch for each of a series of musical tones. Timing is indicated by the vertical position of the bottom edge of a note. The vertical length of a note indicates duration. Pitch is indicated by the horizontal position of a note. Piano roll notation may also be depicted in horizontal, rather than vertical, format.
Many other systems for visually displaying musical data exist. One property common to all such systems is that they are inherently two-dimensional, i.e., the spatial path representing the musical time axis lies exclusively within the substantially two-dimensional plane of the display surface (referred to throughout as the xe2x80x9cimage planexe2x80x9d). While these two-dimensional systems for displaying musical data are adequate for conventional applications such as instructing musicians, two-dimensional systems are not ideal for displaying musical data in the context of an interactive gaming environment, because in the state of the art for interactive gaming three-dimensional graphical environments have become the norm.
The present invention provides a method and apparatus for displaying musical data in a three-dimensional environment in which the spatial path corresponding to the musical time axis does not lie within the two-dimensional image plane onto which the three-dimensional environment is rendered. In one embodiment the perception of the viewer is that of traveling along a path in the three-dimensional environment. The path may be straight or curved and, in some embodiments, the path comprises multiple curved segments. This method of displaying musical data provides a viewer with the appearance of navigating through a three-dimensional space, which is a common paradigm for interactive entertainment software such as game software. Thus, the present invention provides a way to represent musical data in a way that is suitable for entertainment purposes.
In one aspect the present invention relates to a method for displaying musical data on the image plane of a provided display. Musical data is displayed in a three-dimensional graphical environment such that a spatial path corresponding to the musical time axis associated with the musical data does not lie within the substantially two-dimensional image plane of the provided display. In some embodiments that spatial path is linear, curved, continuous, or discontinuous. Musical data may be displayed on graphical surfaces aligned with the spatial path. The graphical surfaces form, in some embodiments, a tunnel through which the time axis associated with the musical data runs. In other embodiments the graphical surfaces form a road along which the time axis associated with the musical data runs. The graphic symbols representing the musical data may be manipulated to provide the viewer with the appearance of motion along the time axis associated with the musical data. Alternatively, the virtual xe2x80x9ccameraxe2x80x9d associated with the thee-dimensional environment may be moved along the time axis associated with the musical data to provide the user with the appearance of motion.
In another aspect, the present invention relates to an apparatus for displaying musical data that includes a display having an image plane and a graphics engine that displays musical data on the display such that a spatial path corresponding to a musical time axis associated with the musical data does not lie in the image plane of the display. In some embodiments, a memory element stores the musical data for display. The graphics engine can display musical data along a spatial path that is linear, non-linear, continuous, or discontinuous. In other embodiments the graphics engine manipulates the graphical symbols associated with the musical data to provide a viewer with the appearance of motion along the time axis associated with the musical data. Alternatively, the graphics engine may change the position of a camera to provide the appearance of motion to the viewer.
In yet another aspect, the present invention relates to a method for displaying musical data on a display having an image plane. A display having an image plane is provided. A time axis associated with musical data is represented as a spatial path through a three-dimensional space. Graphical symbols associated with the musical data are plotted along the spatial axis and the graphical symbols are rendered onto the image plane of the display. In some embodiments, the musical data for display is retrieved from a memory element. In some embodiments that spatial path is linear, curved, continuous, or discontinuous. Musical data may be displayed on graphical surfaces aligned with the spatial path. The graphical surfaces form, in some embodiments, a tunnel through which the time axis associated with the musical data runs. In other embodiments, the graphical surfaces form a road along which the time axis associated with the musical data runs. The graphical symbols representing the musical data may be manipulated to provide the viewer with the appearance of motion along the time axis associated with the musical data. Alternatively, a camera representing the user""s position in the three-dimensional environment may be moved to provide the user with the appearance of motion.